Add second audio channel for voice#169875
Draft
synesthesiam wants to merge 2 commits intodevfrom
Draft
Conversation
synesthesiam
requested review from
arturpragacz,
bdraco,
jesserockz and
kbx81
as code owners
home-assistant
Bot
added
cla-signed
has-tests
integration: assist_pipeline
integration: esphome
new-feature
Top 100
Contributor
|
Hey there @arturpragacz, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
Contributor
|
Hey there @jesserockz, @kbx81, @bdraco, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
synesthesiam
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds multi-channel audio support for ESPHome Assist Satellites by exposing STT audio-processing preferences via the Assist pipeline STT_START event and selecting an “unenhanced” audio channel when the STT provider indicates it prefers no AGC/noise reduction.
Changes:
- Include
audio_processing(from the selected STT provider) in the Assist pipelineSTT_STARTevent payload. - Track multi-channel audio capability in the ESPHome assist satellite and optionally switch to channel 1 based on
STT_STARTpreferences. - Add/adjust tests and snapshots to validate the new event payload and channel-selection behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
homeassistant/components/esphome/assist_satellite.py |
Adds multi-channel capability tracking and audio-channel selection based on STT_START audio processing preferences. |
homeassistant/components/assist_pipeline/pipeline.py |
Emits audio_processing preferences in the STT_START pipeline event. |
tests/components/esphome/test_assist_satellite.py |
Adds coverage for multi-channel selection and channel-0 fallback behavior. |
tests/components/assist_pipeline/snapshots/test_websocket.ambr |
Updates websocket snapshots to include audio_processing in STT_START data. |
tests/components/assist_pipeline/snapshots/test_init.ambr |
Updates init snapshots to include audio_processing in STT_START data. |
Comment on lines
+581
to
593
| # Default to enhanced audio (channel 0) | ||
| active_data = data | ||
|
|
||
| if ( | ||
| self._has_multi_channel_audio | ||
| and (data2 is not None) | ||
| and (self._active_audio_channel == 1) | ||
| ): | ||
| # Non-enhanced audio (channel 1) | ||
| active_data = data2 | ||
|
|
||
| self._audio_queue.put_nowait(active_data) | ||
|
|
Comment on lines
+2351
to
+2359
| async with asyncio.timeout(1): | ||
| await satellite.handle_pipeline_start( | ||
| conversation_id="", | ||
| flags=VoiceAssistantCommandFlag(0), # stt | ||
| audio_settings=VoiceAssistantAudioSettings(), | ||
| wake_word_phrase=None, | ||
| ) | ||
| await satellite.handle_audio(b"channel 0", b"channel 1") | ||
| await satellite.handle_pipeline_stop(abort=False) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Breaking change
Proposed change
NOTE: Requires an updated aioesphomeapi that hasn't been released yet
Adds a second audio channel to voice API audio. This allows for providing the best audio for different parts of the voice pipeline: home-assistant/architecture#1364
This code assumes that channel 0 (the default) contains "enhanced" audio with automatic gain control, noise suppression, etc. and that channel 1 (new) contains "unenhanced" audio.
When the voice assistant has the
MULTI_CHANNEL_AUDIOfeature flag, a second "unenhanced" audio channel may be preset asdata2in thehandle_audiomethod of the ESPHome Assist Satellite entity. TheSTT_STARTvoice event is checked for the audio preferences of the pipeline's speech-to-text entity. If thesttentity prefers not to have automatic gain control and noise suppression, then channel 1 is selected for the pipeline run.Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: